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(54) HUMAN BODY RECOGNITION SENSOR AND NON-CONTACT OPERATION DEVICE 

(57)Abstract: 

PURPOSE: To obtain a human body operation 
recognition sensor sensing an infrared ray emitted from 
a human body to recognize the operation of the human 
body and the non-contact operation device available for 
operation control of various electronic devices using 
the sensor. 

CONSTITUTION: A sensor 11 senses a human body or 
its part and provides the result as an output of image 
information, a characteristic extract device 12 
discriminates the shape of the human body or its part 
and/or its state based on the image information 
outputted from the sensor 11, an operation recognition 
device 13 recognizes and discriminates the operation of 
the human body or its part based on a timewise change 
of shape information outputted from the characteristic 
extract device 12, which provides an output of human 
body operation information. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]A human body action recognition sensor comprising: 
A detection apparatus which detects a human body or its part. 

A feature extraction device which distinguishes said human body, or some [ the ] shape and/or 
situations from picture information outputted from said detection apparatus. 
An operation recognition device which carries out recognition distinction of the operation of 
said human body or its part from a temporal change of form information outputted from said 
feature extraction device. 

[Claim 2]The human body action recognition sensor according to claim 1 which is a detection 
apparatus with which a detection apparatus detects infrared rays emitted from a human body. 
[Claim 3]The human body action recognition sensor according to claim 1 or 2 which it is in a 
gap which wants to choose a detection apparatus from a pyroelectric infrared array sensor, a 
pyroelectric infrared matrix sensor, pyroelectric infrared vidicon, and a semiconductor type 
infrared sensor. 

[Claim 4]A feature extraction device which distinguishes said human body, or some [ the ] 
shape and/or situations from picture information outputted from a detection apparatus which 
detects a human body or its part, and said detection apparatus, An operation recognition device 
which carries out recognition distinction of the operation of said human body or its part from a 
temporal change of form information outputted from said feature extraction device, A 
noncontact manual operating device possessing a human body action pattern recognition device 
which outputs a manipulate signal which compares an operating procedure of electronic 
equipment beforehand decided to be the human body action information recognized by said 
operation recognition device, and corresponds. 

[Claim 5]The noncontact manual operating device according to claim 4 connected with two or 
more electronic equipment via a communication apparatus which has a remote control 
communication apparatus and was formed in the electronic equipment side. 
[Claim 6]The noncontact manual operating device according to claim 5 which chooses one 
operation target of two or more electronic equipment to be operated from operation of a human 
body or its part. 

[Claim 7]The noncontact manual operating device according to any one of claims 4 to 6 which is 
a detection apparatus with which a detection apparatus detects infrared rays emitted from a 
human body. 

[Claim 8]The noncontact manual operating device according to any one of claims 4 to 7 which it 
is in a gap which wants to choose a detection apparatus from a pyroelectric infrared array 
sensor, a pyroelectric infrared matrix sensor, pyroelectric infrared vidicon, and a semiconductor 
type infrared sensor. 

[Claim 9]The noncontact manual operating device according to claim 4 whose electronic 
equipment to be operated is a computer. 

[Claim 10]Electronic equipment to be operated Stereo amplifier, a stereo tuner, a cassette 
deck, The noncontact manual operating device according to claim 5 or 6 which is a household- 
electronic-equipment system containing at least one chosen from a compact disk player, 
television, a videotape recorder, and an optical disk player. 
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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Industrial Application]This invention relates to noncontact manual operating devices, such as 
electronic equipment using the human body action recognition sensor and it which detect the 
infrared rays etc. which are emitted from a human body, and recognize operation of a human 
body or its part. 
[0002] 

[Description of the Prior Art]It is in the tendency which the function of various electronic 
equipment increases, and the number of the switches installed in the distribution power board of 
each electronic equipment increases by in recent years with development of electronics, or an 
operation method complicates. In particular, in consumer electronics, remote control 
instrumentation of a distribution power board progresses, for example, the remote control 
device for exclusive use is attached to each model, such as television, video, audio equipment, 
and an air-conditioner. 
[0003] 

[Problem(s) to be Solved by the Invention]However, since arrangement of an operation body 
system or switches differs according to a use for every model even if it is a product of the 
maker same not to mention the case where the makers of each consumer electronics differ, 
While it was impossible to have communalized a remote control device as a matter of fact, 
therefore many remote control devices were scattered about in the home, the work which 
masters the operation method of each apparatus for a user had become a big burden. Since it 
was mainly designed for the healthy person as a user in conventional apparatus, it was not easy 
to operate those apparatus for people who have a handicap in elderly people, a child, or the 
body. 

[0004]This invention is made in order to solve the above problems, and it is a thing. 
It is providing the noncontact manual operating device of the electronic equipment using the 
target human body action recognition sensor and this which detect infrared rays and recognize 
operation of a human body. 

[0005] 

[Means for Solving the Problem]In order to attain the above-mentioned purpose, a human body 
action recognition sensor of this invention, A feature extraction device which distinguishes said 
human body, or some [ the ] shape and/or situations from picture information outputted from a 
detection apparatus which detects a human body or its part, and said detection apparatus, It is 
constituted so that an operation recognition device which carries out recognition distinction of 
the operation of said human body or its part from a temporal change of form information 
outputted from said feature extraction device may be provided. As for a detection apparatus, in 
the above-mentioned composition, it is preferred that it is a detection apparatus which detects 
infrared rays emitted from a human body. As for a detection apparatus, in the above-mentioned 
composition, it is preferred that it is in a gap to be chosen from a pyroelectric infrared array 
sensor, a pyroelectric infrared matrix sensor, pyroelectric infrared vidicon, and a semiconductor 
type infrared sensor. 

[0006]A detection apparatus with which a noncontact manual operating device of this invention, 



on the other hand, detects a human body or its part, A feature extraction device which 
distinguishes said human body, or some [ the ] shape and/or situations from picture information 
outputted from said detection apparatus, An operation recognition device which carries out 
recognition distinction of the operation of said human body or its part from a temporal change 
of form information outputted from said feature extraction device, It is constituted so that a 
human body action pattern recognition device which outputs a manipulate signal which 
compares an operating procedure of electronic equipment beforehand decided to be the human 
body action information recognized by said operation recognition device, and corresponds may 
be provided. In the above-mentioned composition, it is preferred to be connected with two or 
more electronic equipment via a communication apparatus which has a remote control 
communication apparatus and was formed in the electronic equipment side. In the above- 
mentioned composition, it is preferred to choose one operation target of two or more electronic 
equipment to be operated from operation of a human body or its part. As for a detection 
apparatus, in the above-mentioned composition, it is preferred that it is a detection apparatus 
which detects infrared rays emitted from a human body. As for a detection apparatus, in the 
above-mentioned composition, it is preferred that it is in a gap to be chosen from a pyroelectric 
infrared array sensor, a pyroelectric infrared matrix sensor, pyroelectric infrared vidicon, and a 
semiconductor type infrared sensor. As for electronic equipment to be operated, in the above- 
mentioned composition, it is preferred that it is a computer. In the above-mentioned 
composition, electronic equipment to be operated, It is preferred that it is a household- 
electronic-equipment system containing at least one chosen from stereo amplifier, a stereo 
tuner, a cassette deck, a compact disk player, television, a videotape recorder, and an optical 
disk player. 
[0007] 

[Function]In the human body action recognition sensor of this invention constituted as 
mentioned above. For example, the infrared rays emitted from a human body or its part by the 
detection apparatus which comprised infrared sensing elements, such as a pyroelectric infrared 
array sensor, a pyroelectric infrared matrix sensor, pyroelectric infrared vidicon, and a 
semiconductor type infrared sensor, are detected, and it outputs as picture information. A 
feature extraction device distinguishes a human body, or some [ the ] shape and/or situations 
from the picture information outputted from a detection apparatus. An operation recognition 
device carries out recognition distinction of the operation of a human body or its part from the 
temporal change of the form information outputted from a feature extraction device. As a result, 
operation of a human body or its part can be recognized. 

[0008]In the noncontact manual operating device of this invention constituted as mentioned 
above. Since the manipulate signal with which a human body action pattern recognition device 
compares the operating procedure of the electronic equipment beforehand decided to be the 
human body action information recognized by the operation recognition device, and furthermore 
corresponds is outputted, It becomes possible to operate electronic equipment, such as a 
computer and TV-for-home JON, for example, without touching a final controlling element 
directly. By connecting with two or more electronic equipment via the communication apparatus 
formed in the remote control communication apparatus and electronic equipment side, and 
choosing one operation target of two or more electronic equipment to be operated from 
operation of a human body or its part, It becomes possible to operate the home audio system 
which contains two or more electronic equipment, for example, stereo amplifier, a stereo tuner, 
a cassette deck, and a compact disk player with one noncontact manual operating device. 
[0009] 
[Example] 

(Example 1) The 1st suitable example of the human body action recognition sensor of this 
invention and the non-** type manual operating device of the electronic equipment using it is 
described, referring to drawing 1 and drawing 2 . Drawing 1 is a block diagram showing the 
composition of the human body action recognition sensor of the 1st example, and drawing 2 is a 
block diagram showing the composition of the noncontact manual operating device of the 
electronic equipment using the human body action recognition sensor shown in drawing 1 . In 
this example, the form of a hand and operation of a hand shall be recognized as a recognition 



object of a human body action recognition sensor. 

[0010]In drawing 1 , the human body action recognition sensor 10 possesses the detection 
apparatus 11, the feature extraction device 12, and the operation recognition device 13. The 
detection apparatus 1 1 detects a human body or its infrared rays emitted from a part (for 
example, hand), It is for outputting the infrared image information, for example, infrared imaging 
apparatus, such as a pyroelectric infrared array sensor, a pyroelectric infrared matrix sensor, 
pyroelectric infrared vidicon, and a semiconductor type infrared sensor, are used. The detection 
apparatus 11 samples the intensity etc. of the infrared rays which each sensor element detects 
synchronizing with a clock signal etc., and outputs the signal corresponding to the intensity of 
the infrared rays, etc. For example, when a matrix sensor is used as the detection apparatus 11, 
each sensor element which counters a hand outputs the signal equivalent to "1" which detects 
the infrared rays emitted from a hand and is said with a predetermined signal, for example, a 
digital signal. On the other hand, since intensity is small even if a hand and the sensor element 
which has not countered do not detect the infrared rays emitted from a hand or detects them, 
the signal equivalent to "0" said with another predetermined signal, for example, a digital signal, 
is outputted. The feature extraction device 1 2 distinguishes a human body or a part of [ the ] 
shape from the picture information outputted from the detection apparatus 11, and outputs 
them as form information. For example, it can be known whether a hand is in what kind of shape 
or situations (for example, good one of janken, CHOKI, a par, etc.) by following the address 
(position) of the sensor which is outputting the signal equivalent to aforementioned "1", and 
going. The operation recognition device 13 recognizes operation of a human body or its part 
from the form information outputted from the feature extraction device 12, and its temporal 
change, and outputs it as human body action information. For example, the output from the 
feature extraction device 12 at the time of a certain sampling and the output at the time of the 
next sampling are measured, and it is got to know how the address (position) of the sensor 
which is outputting the signal equivalent to aforementioned "1" moved or changed. Thereby, the 
hand's having moved the front of the detection apparatus 1 1 to the left from the right, for 
example and the human body action that the shape of the hand changed from Gout to the par 
can be recognized. 

[0011]In drawing 2 , the noncontact manual operating device 20 of this invention possesses the 
above-mentioned human body action recognition sensor 10 and the human body action pattern 
recognition device 21, and is connected to the electronic equipment 22. The human body action 
information outputted from the human body action recognition sensor 10 is inputted into the 
human body action pattern recognition device 21, compares the combination of a human body 
action pattern and the operation of the electronic equipment 22 on which it decided artificially 
beforehand, outputs a corresponding manipulate signal, and controls the electronic equipment 
22. For example, TV-for— home JON is connected to the noncontact manual operating device 20 
as the electronic equipment 22, and the case where the switching control of a receiving channel 
and the sound volume increase and decrease of operation are performed is considered. For 
example, when the shape of a hand is good, switching operation of a receiving channel is 
performed, and when the shape of a hand is a par, it decides to perform the increase and 
decrease of operation of sound volume. When a hand moves the front face of the sensing 
device 1 1 to the left from the right, it decreases a receiving channel or sound volume at a time 
by one step, and when a hand moves to the right from the left, it decides to make it increase 
one step at a time. The outstanding operativity is acquired without touching the final controlling 
element of TV-for-home JON directly by constituting in this way. 

[0012]It explains referring to drawing 3 which is a block diagram showing the outline composition 
for (Example 2), next the 2nd suitable example of the noncontact manual operating device of 
this invention. In drawing 3 , the noncontact manual operating device 30 possesses the human 
body action recognition sensor 10, the human body action pattern recognition device 21, and 
the remote control communication apparatus 31, and is connected to two or more electronic 
equipment 331 -33n via the apparatus side communication apparatus 321 -32n. That is, it makes 
it possible to carry out operation control of two or more electronic equipment 331 -33n with the 
one noncontact manual operating device 30 by forming the remote control communication 
apparatus 31 and the apparatus side communication apparatus 321 -32n. Since the human body 



action recognition sensor 10 and the human body action pattern recognition device 21 are the 
same as that of what was shown in the 1st example of the above, the explanation is omitted. 
[0013]The case where the home audio system which comprised stereo amplifier, a stereo tuner, 
a cassette deck, a compact disk player, etc., for example is operated as the electronic 
equipment 331~33n is considered. For example, the difference in the number of a finger is used 
as a method of identifying the electronic equipment 331 -33n to be operated, for example, such 
as assigning the state where one finger was upraised to stereo amplifier, and assigning the state 
where two fingers were upraised to a stereo tuner. First, where two fingers are upraised, a hand 
is brought to the front face of the detection apparatus 1 1 shown in drawing 1 . It recognizes that 
two fingers stand as for the human body action recognition sensor 10, and since two fingers 
stand as for the human body action pattern recognition device, it chooses a stereo tuner as 
music reproduction sauce. Next, it moves one step of tuning of a stereo tuner at a time by 
changing a hand into a good state and moving the front face of the detection apparatus 1 1 to 
the right from the left. This operation is repeated until a predetermined broadcasting station is 
chosen. Next, where only one finger is upraised, a hand is brought to the front face of the 
detection apparatus 1 1. Since one finger stands as for the human body action recognition 
sensor 10 and the human body action pattern recognition device, they choose stereo amplifier 
as operation equipment. And a hand is changed into a par state, and you move the front face of 
the detection apparatus 1 1 to the right or the left from the right from the left, and make it 
fluctuate one step of volume volume of stereo amplifier at a time. Without touching the final 
controlling element of an audio system by constituting in this way, the switching operation of 
music reproduction sauce, operation of each apparatus, and the volume volume increase and 
decrease of operation can be performed, and the outstanding operativity is acquired. Although 
this example explained the case where the home audio system which comprised stereo 
amplifier, a stereo tuner, a cassette deck, a compact disk player, etc. was operated, There is 
not necessarily necessity of including all these components, and even if it is the system which 
combined arbitrarily selected apparatus, it can be operated similarly. It is a home audiovisual 
system containing television, a videotape recorder, an optical disk player, etc. other than these, 
and can be operated similarly. 

[0014](Example 3), next the 3rd example connected to the computer 40 by using the 
noncontact manual operating device of this invention as an input device are shown in drawing 4 . 
In drawing 4 , the human body action recognition sensor 10 and the human body action pattern 
recognition device 21 are the same as that of the thing of the 1st example shown in drawing 2 . 
A motion of a user s hand can be made to correspond to the emulation operation of pointing 
devices, such as a mouse and a trackball, and a keyboard by constituting in this way, Operation 
of the computer 40 is attained by non-contact, and the beginner unfamiliar to a keyboard can 
also master operation in a short time. 

[0015]Operations of opening and closing of the form of an arm, a motion of an eyeball, a 
palpebra, and a mouth, etc. and those combination may be recognized, and although the motion 
of a hand was made into the recognition object as some human bodies, it may constitute from 
each above-mentioned example so that it may be made to correspond with the operating 
procedure of electronic equipment. 
[0016] 

[Effect of the Invention]according to the human body action recognition sensor of this invention 
as mentioned above — a detection apparatus — a human body — or — the — a part being 
detected, and it outputting as picture information, and, A human body, or some [ the ] shape 
and/or situations are distinguished from the picture information outputted from a detection 
apparatus by a feature extraction device, Since it constituted so that recognition distinction of 
the operation of a human body or its part might be carried out from the temporal change of the 
form information outputted from a feature extraction device by an operation recognition device, 
operation of a human body or its part can be recognized. By constituting a detection apparatus, 
for example from infrared sensing elements, such as a pyroelectric infrared array sensor, a 
pyroelectric infrared matrix sensor, pyroelectric infrared vidicon, and a semiconductor type 
infrared sensor, Operation of a human body or its part can be recognized from a human body or 
its part using the infrared rays emitted. 



[0017]According to the noncontact manual operating device of this invention constituted as 
mentioned above. A human body, or some [ the ] shape and/or situations are distinguished from 
a human body or its picture information which detects a part, outputs as picture information and 
is outputted from a detection apparatus by a feature extraction device with a detection 
apparatus, Recognition distinction of the operation of a human body or its part is carried out 
from the temporal change of the form information outputted from a feature extraction device by 
an operation recognition device, Since it constituted so that the manipulate signal which 
compares the operating procedure of the electronic equipment beforehand decided to be the 
human body action information recognized by the operation recognition device with the human 
body action pattern recognition device, and corresponds might be outputted, It becomes 
possible to operate electronic equipment, such as a computer and TV-for-home JON, for 
example, without touching a final controlling element directly. By connecting with two or more 
electronic equipment via the communication apparatus formed in the remote control 
communication apparatus and electronic equipment side, and choosing one operation target of 
two or more electronic equipment to be operated from operation of a human body or its part, 
With one noncontact manual operating device, two or more electronic equipment, for example, 
stereo amplifier, It becomes possible to operate the household-electronic-equipment systems 
(home audio system etc.) containing at least one of a stereo tuner, a cassette deck, a compact 
disk player, television, a videotape recorder, optical disk players, etc. By constituting a 
detection apparatus, for example from infrared sensing elements, such as a pyroelectric infrared 
array sensor, a pyroelectric infrared matrix sensor, pyroelectric infrared vidicon, and a 
semiconductor type infrared sensor, Operation of a human body or its part can be recognized 
from a human body or its part using the infrared rays emitted. 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The block diagram showing the composition of the 1st example of the human body 
action recognition sensor of this invention 

[Drawing 2] The block diagram showing the composition of the 1st example of the noncontact 
manual operating device of this invention using the human body action recognition sensor 
shown in drawing 1 

[Drawing 3] The block diagram showing the composition of the 2nd example of the noncontact 
manual operating device of this invention 

[Drawing 4] The block diagram showing the composition of the 3rd example of the noncontact 

manual operating device of this invention 

[Description of Notations] 

10: Human body action recognition sensor 

11: Detection apparatus 

12: Feature extraction device 

13: Operation recognition device 

20: Noncontact manual operating device 

21: Human body action pattern recognition device 

22: Electronic equipment 

30: Noncontact manual operating device 

31: Remote control communication apparatus 

321 -32n: Apparatus side communication apparatus 

331 -33n: Electronic equipment 

40: Computer 
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[Drawing 1] 
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